
<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta charset="utf-8" />
  
  <!-- Licensed under the Apache 2.0 License -->
  <link rel="stylesheet" type="text/css" href="../_static/fonts/open-sans/stylesheet.css" />
  <!-- Licensed under the SIL Open Font License -->
  <link rel="stylesheet" type="text/css" href="../_static/fonts/source-serif-pro/source-serif-pro.css" />
  <link rel="stylesheet" type="text/css" href="../_static/css/bootstrap.min.css" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
    <title>Experiments &#8212; jMetalPy 1.5.3 documentation</title>
    <link rel="stylesheet" href="../_static/guzzle.css" type="text/css" />
    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
    <link rel="stylesheet" type="text/css" href="../_static/custom.css" />
    <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
    <script type="text/javascript" src="../_static/jquery.js"></script>
    <script type="text/javascript" src="../_static/underscore.js"></script>
    <script type="text/javascript" src="../_static/doctools.js"></script>
    <script type="text/javascript" src="../_static/language_data.js"></script>
    <script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
    <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
    <script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["$", "$"], ["\\(", "\\)"]], "processEscapes": true, "ignoreClass": "document", "processClass": "math|output_area"}})</script>
    <link rel="index" title="Index" href="../genindex.html" />
    <link rel="search" title="Search" href="../search.html" />
    <link rel="next" title="Statistical analysis" href="statistics.html" />
    <link rel="prev" title="Front visualization" href="visualization.html" />
  
   

  </head><body>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="../genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="../py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="statistics.html" title="Statistical analysis"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="visualization.html" title="Front visualization"
             accesskey="P">previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="../index.html">jMetalPy 1.5.3 documentation</a> &#187;</li>
          <li class="nav-item nav-item-1"><a href="../tutorials.html" accesskey="U">Getting started</a> &#187;</li> 
      </ul>
    </div>
    <div class="container-wrapper">

      <div id="mobile-toggle">
        <a href="#"><span class="glyphicon glyphicon-align-justify" aria-hidden="true"></span></a>
      </div>
  <div id="left-column">
    <div class="sphinxsidebar"><a href="
    ../index.html" class="text-logo">
    <img src="_static/jmetalpy.png" class="img-fluid" alt="jMetalPy 1.5.3 documentation">
    <br>
</a>

<div class="sidebar-block">
  <div class="sidebar-wrapper">
    Python version of the jMetal framework
  </div>
</div>
<div class="sidebar-block">
  <div class="sidebar-wrapper">
    <h2>Table Of Contents</h2>
  </div>
  <div class="sidebar-toc">
    
    
      <ul class="current">
<li class="toctree-l1 current"><a class="reference internal" href="../tutorials.html">Getting started</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="problem.html">Defining new problems</a></li>
<li class="toctree-l2"><a class="reference internal" href="observer.html">Extending algorithms</a></li>
<li class="toctree-l2"><a class="reference internal" href="evaluator.html">Evaluate solutions</a></li>
<li class="toctree-l2"><a class="reference internal" href="visualization.html">Front visualization</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Experiments</a></li>
<li class="toctree-l2"><a class="reference internal" href="statistics.html">Statistical analysis</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../multiobjective.algorithms.html">Multi-objective algorithms</a></li>
<li class="toctree-l1"><a class="reference internal" href="../singleobjective.algorithms.html">Single-objective algorithms</a></li>
<li class="toctree-l1"><a class="reference internal" href="../operators.html">Operators</a></li>
<li class="toctree-l1"><a class="reference internal" href="../problems.html">Problems</a></li>
</ul>

    
  </div>
</div>
<div class="sidebar-block">
  <div class="sidebar-wrapper">
    <h2>Contents</h2>
    <div class="sidebar-localtoc">
      <ul>
<li><a class="reference internal" href="#">Experiments</a><ul>
<li><a class="reference internal" href="#running-the-experiment">Running the experiment</a></li>
<li><a class="reference internal" href="#summary-file">Summary file</a></li>
<li><a class="reference internal" href="#module-jmetal.lab.experiment">API</a></li>
</ul>
</li>
</ul>

    </div>
  </div>
</div>
<div class="sidebar-block">
  <div class="sidebar-wrapper">
    <div id="main-search">
      <form class="form-inline" action="../search.html" method="GET" role="form">
        <div class="input-group">
          <input name="q" type="text" class="form-control" placeholder="Search...">
        </div>
        <input type="hidden" name="check_keywords" value="yes" />
        <input type="hidden" name="area" value="default" />
      </form>
    </div>
  </div>
</div>
      
    </div>
  </div>
        <div id="right-column">
          
          <nav aria-label="breadcrumb">
            <ol class="breadcrumb">
              <li class="breadcrumb-item"><a href="../index.html">Docs</a></li>
              
              <li class="breadcrumb-item"><a href="../tutorials.html">Getting started</a></li>
              
              <li class="breadcrumb-item">Experiments</li>
            </ol>
          </nav>
          
          <div class="document clearer body">
            
  <div class="section" id="experiments">
<h1>Experiments<a class="headerlink" href="#experiments" title="Permalink to this headline">¶</a></h1>
<div class="section" id="running-the-experiment">
<h2>Running the experiment<a class="headerlink" href="#running-the-experiment" title="Permalink to this headline">¶</a></h2>
<p>This is an example of an experimental study based on solving three problems of the ZDT family with three
different algorithms: NSGA-II, GDE3 and SMPSO.</p>
<p>The hypervolume, generational distance and epsilon indicators are used for performance assessment.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">jmetal.algorithm.multiobjective.gde3</span> <span class="kn">import</span> <span class="n">GDE3</span>
<span class="kn">from</span> <span class="nn">jmetal.algorithm.multiobjective.nsgaii</span> <span class="kn">import</span> <span class="n">NSGAII</span>
<span class="kn">from</span> <span class="nn">jmetal.algorithm.multiobjective.smpso</span> <span class="kn">import</span> <span class="n">SMPSO</span>
<span class="kn">from</span> <span class="nn">jmetal.core.quality_indicator</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">jmetal.lab.experiment</span> <span class="kn">import</span> <span class="n">Experiment</span><span class="p">,</span> <span class="n">Job</span><span class="p">,</span> <span class="n">generate_summary_from_experiment</span>
<span class="kn">from</span> <span class="nn">jmetal.operator</span> <span class="kn">import</span> <span class="n">PolynomialMutation</span><span class="p">,</span> <span class="n">SBXCrossover</span>
<span class="kn">from</span> <span class="nn">jmetal.problem</span> <span class="kn">import</span> <span class="n">ZDT1</span><span class="p">,</span> <span class="n">ZDT2</span><span class="p">,</span> <span class="n">ZDT3</span>
<span class="kn">from</span> <span class="nn">jmetal.util.archive</span> <span class="kn">import</span> <span class="n">CrowdingDistanceArchive</span>
<span class="kn">from</span> <span class="nn">jmetal.util.termination_criterion</span> <span class="kn">import</span> <span class="n">StoppingByEvaluations</span>


<span class="k">def</span> <span class="nf">configure_experiment</span><span class="p">(</span><span class="n">problems</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">n_run</span><span class="p">:</span> <span class="nb">int</span><span class="p">):</span>
    <span class="n">jobs</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">max_evaluations</span> <span class="o">=</span> <span class="mi">25000</span>

    <span class="k">for</span> <span class="n">run</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n_run</span><span class="p">):</span>
        <span class="k">for</span> <span class="n">problem_tag</span><span class="p">,</span> <span class="n">problem</span> <span class="ow">in</span> <span class="n">problems</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
            <span class="n">jobs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
                <span class="n">Job</span><span class="p">(</span>
                    <span class="n">algorithm</span><span class="o">=</span><span class="n">NSGAII</span><span class="p">(</span>
                        <span class="n">problem</span><span class="o">=</span><span class="n">problem</span><span class="p">,</span>
                        <span class="n">population_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span>
                        <span class="n">offspring_population_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span>
                        <span class="n">mutation</span><span class="o">=</span><span class="n">PolynomialMutation</span><span class="p">(</span><span class="n">probability</span><span class="o">=</span><span class="mf">1.0</span> <span class="o">/</span> <span class="n">problem</span><span class="o">.</span><span class="n">number_of_variables</span><span class="p">,</span>
                                                    <span class="n">distribution_index</span><span class="o">=</span><span class="mi">20</span><span class="p">),</span>
                        <span class="n">crossover</span><span class="o">=</span><span class="n">SBXCrossover</span><span class="p">(</span><span class="n">probability</span><span class="o">=</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">distribution_index</span><span class="o">=</span><span class="mi">20</span><span class="p">),</span>
                        <span class="n">termination_criterion</span><span class="o">=</span><span class="n">StoppingByEvaluations</span><span class="p">(</span><span class="nb">max</span><span class="o">=</span><span class="n">max_evaluations</span><span class="p">)</span>
                    <span class="p">),</span>
                    <span class="n">algorithm_tag</span><span class="o">=</span><span class="s1">&#39;NSGAII&#39;</span><span class="p">,</span>
                    <span class="n">problem_tag</span><span class="o">=</span><span class="n">problem_tag</span><span class="p">,</span>
                    <span class="n">run</span><span class="o">=</span><span class="n">run</span><span class="p">,</span>
                <span class="p">)</span>
            <span class="p">)</span>
            <span class="n">jobs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
                <span class="n">Job</span><span class="p">(</span>
                    <span class="n">algorithm</span><span class="o">=</span><span class="n">GDE3</span><span class="p">(</span>
                        <span class="n">problem</span><span class="o">=</span><span class="n">problem</span><span class="p">,</span>
                        <span class="n">population_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span>
                        <span class="n">cr</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span>
                        <span class="n">f</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span>
                        <span class="n">termination_criterion</span><span class="o">=</span><span class="n">StoppingByEvaluations</span><span class="p">(</span><span class="nb">max</span><span class="o">=</span><span class="n">max_evaluations</span><span class="p">)</span>
                    <span class="p">),</span>
                    <span class="n">algorithm_tag</span><span class="o">=</span><span class="s1">&#39;GDE3&#39;</span><span class="p">,</span>
                    <span class="n">problem_tag</span><span class="o">=</span><span class="n">problem_tag</span><span class="p">,</span>
                    <span class="n">run</span><span class="o">=</span><span class="n">run</span><span class="p">,</span>
                <span class="p">)</span>
            <span class="p">)</span>
            <span class="n">jobs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
                <span class="n">Job</span><span class="p">(</span>
                    <span class="n">algorithm</span><span class="o">=</span><span class="n">SMPSO</span><span class="p">(</span>
                        <span class="n">problem</span><span class="o">=</span><span class="n">problem</span><span class="p">,</span>
                        <span class="n">swarm_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span>
                        <span class="n">mutation</span><span class="o">=</span><span class="n">PolynomialMutation</span><span class="p">(</span><span class="n">probability</span><span class="o">=</span><span class="mf">1.0</span> <span class="o">/</span> <span class="n">problem</span><span class="o">.</span><span class="n">number_of_variables</span><span class="p">,</span>
                                                    <span class="n">distribution_index</span><span class="o">=</span><span class="mi">20</span><span class="p">),</span>
                        <span class="n">leaders</span><span class="o">=</span><span class="n">CrowdingDistanceArchive</span><span class="p">(</span><span class="mi">100</span><span class="p">),</span>
                        <span class="n">termination_criterion</span><span class="o">=</span><span class="n">StoppingByEvaluations</span><span class="p">(</span><span class="nb">max</span><span class="o">=</span><span class="n">max_evaluations</span><span class="p">)</span>
                    <span class="p">),</span>
                    <span class="n">algorithm_tag</span><span class="o">=</span><span class="s1">&#39;SMPSO&#39;</span><span class="p">,</span>
                    <span class="n">problem_tag</span><span class="o">=</span><span class="n">problem_tag</span><span class="p">,</span>
                    <span class="n">run</span><span class="o">=</span><span class="n">run</span><span class="p">,</span>
                <span class="p">)</span>
            <span class="p">)</span>

    <span class="k">return</span> <span class="n">jobs</span>


<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="c1"># Configure the experiments</span>
    <span class="n">jobs</span> <span class="o">=</span> <span class="n">configure_experiment</span><span class="p">(</span><span class="n">problems</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;ZDT1&#39;</span><span class="p">:</span> <span class="n">ZDT1</span><span class="p">(),</span> <span class="s1">&#39;ZDT2&#39;</span><span class="p">:</span> <span class="n">ZDT2</span><span class="p">(),</span> <span class="s1">&#39;ZDT3&#39;</span><span class="p">:</span> <span class="n">ZDT3</span><span class="p">()},</span> <span class="n">n_run</span><span class="o">=</span><span class="mi">31</span><span class="p">)</span>

    <span class="c1"># Run the study</span>
    <span class="n">output_directory</span> <span class="o">=</span> <span class="s1">&#39;data&#39;</span>
    <span class="n">experiment</span> <span class="o">=</span> <span class="n">Experiment</span><span class="p">(</span><span class="n">output_dir</span><span class="o">=</span><span class="n">output_directory</span><span class="p">,</span> <span class="n">jobs</span><span class="o">=</span><span class="n">jobs</span><span class="p">)</span>
    <span class="n">experiment</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="section" id="summary-file">
<h2>Summary file<a class="headerlink" href="#summary-file" title="Permalink to this headline">¶</a></h2>
<p>The results of this experiment can be summarized to a CSV file as follows:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="c1"># experiment = ...</span>

    <span class="c1"># Generate summary file</span>
    <span class="n">generate_summary_from_experiment</span><span class="p">(</span>
        <span class="n">input_dir</span><span class="o">=</span><span class="n">output_directory</span><span class="p">,</span>
        <span class="n">reference_fronts</span><span class="o">=</span><span class="s1">&#39;/home/user/jMetalPy/resources/reference_front&#39;</span><span class="p">,</span>
        <span class="n">quality_indicators</span><span class="o">=</span><span class="p">[</span><span class="n">GenerationalDistance</span><span class="p">(),</span> <span class="n">EpsilonIndicator</span><span class="p">(),</span> <span class="n">HyperVolume</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">])]</span>
    <span class="p">)</span>
</pre></div>
</div>
<p>This file contains all the information of the quality indicator values, for each configuration and run.
The summary file is the input of all the statistical tests, so that they can be applied to <strong>any valid file having the proper format</strong>.</p>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$</span> head QualityIndicatorSummary.csv
<span class="go">Algorithm,Problem,IndicatorName,ExecutionId,IndicatorValue</span>
<span class="go">NSGAII,ZDT1,EP,0,0.015705992620067832</span>
<span class="go">NSGAII,ZDT1,EP,1,0.012832504015918067</span>
<span class="go">...</span>
</pre></div>
</div>
</div>
<div class="section" id="module-jmetal.lab.experiment">
<span id="api"></span><h2>API<a class="headerlink" href="#module-jmetal.lab.experiment" title="Permalink to this headline">¶</a></h2>
<dl class="class">
<dt id="jmetal.lab.experiment.Experiment">
<em class="property">class </em><code class="sig-prename descclassname">jmetal.lab.experiment.</code><code class="sig-name descname">Experiment</code><span class="sig-paren">(</span><em class="sig-param">output_dir: str, jobs: List[jmetal.lab.experiment.Job], m_workers: int = 6</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/jmetal/lab/experiment.html#Experiment"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#jmetal.lab.experiment.Experiment" title="Permalink to this definition">¶</a></dt>
<dd><p>Bases: <a class="reference external" href="https://docs.python.org/3/library/functions.html#object" title="(in Python v3.8)"><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></a></p>
<dl class="method">
<dt id="jmetal.lab.experiment.Experiment.run">
<code class="sig-name descname">run</code><span class="sig-paren">(</span><span class="sig-paren">)</span> &#x2192; None<a class="reference internal" href="../_modules/jmetal/lab/experiment.html#Experiment.run"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#jmetal.lab.experiment.Experiment.run" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>

</dd></dl>

<dl class="class">
<dt id="jmetal.lab.experiment.Job">
<em class="property">class </em><code class="sig-prename descclassname">jmetal.lab.experiment.</code><code class="sig-name descname">Job</code><span class="sig-paren">(</span><em class="sig-param">algorithm: jmetal.core.algorithm.Algorithm</em>, <em class="sig-param">algorithm_tag: str</em>, <em class="sig-param">problem_tag: str</em>, <em class="sig-param">run: int</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/jmetal/lab/experiment.html#Job"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#jmetal.lab.experiment.Job" title="Permalink to this definition">¶</a></dt>
<dd><p>Bases: <a class="reference external" href="https://docs.python.org/3/library/functions.html#object" title="(in Python v3.8)"><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></a></p>
<dl class="method">
<dt id="jmetal.lab.experiment.Job.execute">
<code class="sig-name descname">execute</code><span class="sig-paren">(</span><em class="sig-param">output_path: str = ''</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/jmetal/lab/experiment.html#Job.execute"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#jmetal.lab.experiment.Job.execute" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>

</dd></dl>

<dl class="function">
<dt id="jmetal.lab.experiment.generate_summary_from_experiment">
<code class="sig-prename descclassname">jmetal.lab.experiment.</code><code class="sig-name descname">generate_summary_from_experiment</code><span class="sig-paren">(</span><em class="sig-param">input_dir: str, quality_indicators: List[jmetal.core.quality_indicator.QualityIndicator], reference_fronts: str = ''</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/jmetal/lab/experiment.html#generate_summary_from_experiment"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#jmetal.lab.experiment.generate_summary_from_experiment" title="Permalink to this definition">¶</a></dt>
<dd><p>Compute a list of quality indicators. The input data directory <em>must</em> met the following structure (this is generated
automatically by the Experiment class):</p>
<ul class="simple">
<li><p>&lt;base_dir&gt;</p>
<ul>
<li><p>algorithm_a</p>
<ul>
<li><p>problem_a</p>
<ul>
<li><p>FUN.0.tsv</p></li>
<li><p>FUN.1.tsv</p></li>
<li><p>VAR.0.tsv</p></li>
<li><p>VAR.1.tsv</p></li>
<li><p>…</p></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>input_dir</strong> – Directory where all the input data is found (function values and variables).</p></li>
<li><p><strong>reference_fronts</strong> – Directory where reference fronts are found.</p></li>
<li><p><strong>quality_indicators</strong> – List of quality indicators to compute.</p></li>
</ul>
</dd>
<dt class="field-even">Returns</dt>
<dd class="field-even"><p>None.</p>
</dd>
</dl>
</dd></dl>

</div>
</div>


          </div>
            
  <div class="footer-relations">
    
      <div class="float-left">
        <a class="btn btn-outline btn-sm" href="visualization.html" title="previous chapter (use the left arrow)">← Front visualization</a>
      </div>
    
      <div class="float-right">
        <a class="btn btn-outline btn-sm" href="statistics.html" title="next chapter (use the right arrow)">Statistical analysis →</a>
      </div>
    </div>
    <div class="clearer"></div>
  
        </div>
        <div class="clearfix"></div>
    </div>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="../genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="../py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="statistics.html" title="Statistical analysis"
             >next</a> |</li>
        <li class="right" >
          <a href="visualization.html" title="Front visualization"
             >previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="../index.html">jMetalPy 1.5.3 documentation</a> &#187;</li>
          <li class="nav-item nav-item-1"><a href="../tutorials.html" >Getting started</a> &#187;</li> 
      </ul>
    </div>
<script type="text/javascript">
  $("#mobile-toggle a").click(function () {
    $("#left-column").toggle();
  });
</script>
<script type="text/javascript" src="../_static/js/bootstrap.js"></script>
  <div class="footer">
    &copy; Copyright 2019, Antonio Benítez-Hidalgo. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
  </div>
  </body>
</html>